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DETAILED ACTION 
Arguments and Amendments 

1. Applicant's amendments and arguments filed 28 February 2005, have been entered and 
made of record. 



Drawings 

2. The corrected figures 1 A and IB have been received. These drawings are accepted. 

Claim Objections 

3. In view of Applicant's amendments to claims 17 and 18, the previous claim objections 
have been withdrawn. 

Response to Arguments 

4. Applicant's arguments with respect to claim 1 have been fully considered but are not 
persuasive for the following reasons. 

Applicant presents three arguments regarding the rejection of claim 1 : 

5. First, Applicant argues that Krishnamachari does not disclose obtaining first and second 
2-D images of an object. Applicant asserts that Krishnamachari' s image retrieval system "does 
not teach, describe or suggest that target image 101 and any of the images 111 (in the retrieval 
database) are of the same object" (see Applicant's Remarks, pp. 12-13). 

It is true that Krishnamachari does not expressly disclose that any of the database images 
are of the same object as the target image. However, the converse is true in that Krishnamachari 
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does not disclose that all the images in the database are not of the same object as the target 
image. The purpose of Krishnamachari's system is to retrieve images from a database that are 
most similar to a target image. 

For example, Krishnamachari's system is operative to store a database of images of 
paintings; the user then provides a target image of a painting. The target image may be an 
unknown painting, and the user wishes identify it; or the user may be familiar with the painting 
and wants to know whether an image of the painting is in the database. Krishnamachari's system 
then compares a color histogram of the target image to a color histogram of each of the images in 
the database. The system then retrieves the images in the database that are most similar to the 
target image based on the comparison. 

Krishnamachari does not expressly state that the target image is of the same object as any 
of the database images. However, such a limitation is so easily inferred as to be implicit in 
Krishnamachari's disclosure. In the above example, the target image and a database image may 
very well be of the same object (i.e. painting). If that is the case, then the system will report that 
the target image and the database image of the same painting are substantially similar. On the 
other hand, if the painting in the target image is not found in any of the database images, then the 
system may not find any of the images in the database to be similar; however, there may be other 
images of paintings in the database that use similar colors or that are by the same artist so that 
the system does find a database image(s) to be similar, although not identical, to the target 
image. Such is the operation of Krishnamachari's system - it does not require that any of the 
database images are of the same object as the target image, but this limitation would have been 
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implicit to those skilled in the art, since the purpose of Krishnamachari's retrieval system is to 
identify any database image(s) that match a provided target image. 

6. Second, Applicant argues that Krishnamachari does not disclose processing a region, 
since Krishnamachari's disclosure is directed to processing entire images (see Applicant's 
Remarks, pp. 14-15). As stated in the previous Office action, Krishnamachari is not considered 
to teach this feature. Therefore, the Moghaddam reference was relied upon to provide motivation 
for processing regions of interest (ROIs) in an image rather than entire images. 

7. Third, Applicant argues that one skilled in the art would not have been motivated to 
combine the teachings of Krishnamachari and Moghaddam to achieve the claimed invention (see 
Applicant's Remarks, pp. 15-17). Examiner maintains that there is sufficient motivation to 
combine the teachings of the two references. Both are related to content-based image retrieval, 
and more specifically, both are related to content-based image retrieval involving the comparison 
of color histograms between images in a database and a target image. 

Applicant asserts that, in contrast to Krishnamachari and Moghaddam, "the present 
invention is used for determining color consistency of regions of images of the same object for 
use in 3-D modeling" (p. 15). Examiner appreciates the differences between the disclosed 
invention and the teachings of Krishnamachari and Moghaddam. However, in the Examiner's 
opinion, the claimed invention is an obvious combination of the prior art image query and 
retrieval systems taught by Krishnamachari and Moghaddam That is, the invention, as claimed 
in claim 1, does not differentiate a method of measuring color consistency for image retrieval 
from a method of measuring color consistency for 3-D modeling. The claim calls for comparing 
the histograms of an object region in two images of the same object, and then processing the 
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region based whether the histograms are similar. In the claim, there are no details regarding how 
the image is "processed" that would differentiate the claim as a method involving 3-D modeling 
from a method involving image retrieval. 

The above remarks regarding claim 1 also apply to the corresponding computer program 
product of claim 19. 

Claim Rejections - 35 USC § 103 

8. The following is a quotation of 35 U.S. C 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

9. Claims 1-3, 5-11, and 19 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
U.S. Patent 6,721,449 by Krishnamachari in view of "Image Retrieval with Local and Spatial 
Queries" by Moghaddam et al. ("Moghaddam"). 

Regarding claims 1 and 19, Krishnamachari discloses a method of measuring color 
consistency comprising: 

obtaining a first 2-D image and a second 2-D image of an object (figure 1 : a target image 
and a reference image of an object from a database are obtained); 

subdividing the first image into a first set of partitions and the second image into a second 
set of image partitions, each partition having a color (i.e. each of the first and second images are 
partitioned into pixels ("partitions"), each pixel having a color); 
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selecting a first subset of image partitions in the first set of image partitions (e.g. a first 
image block as one of 4x4, 8x8, or 16x16 image blocks is a first subset of pixels in the target 
image) and a second subset of image partitions in the second set of image partitions (e.g. a 
second image block as one of 4x4, 8x8, or 16x16 image blocks is a second subset of pixels in the 
reference image) (see column 4, line 59 through column 5, line 13); 

assigning each image partition in the first subset and each image partition in the second 
subset a color value corresponding to the color of the image partition (column 4, lines 33-58: 
pixels in each block are assigned CIELAB or CIELUV color values based on their respective 
colors); 

placing each image partition in the first subset in one of a first series of histogram 
subdivisions and each image partition in the second subset in one of a second series of histogram 
subdivisions based on the color value of each image partition (column 5, lines 14-34: for each of 
the first and second image blocks, an histogram is generated whereby pixels in each block are 
accumulated into histogram subdivisions based on color values of the pixels); 

comparing the first series of histogram subdivisions to the second series of histogram 
subdivisions (column 5, lines 25-34: the histograms of corresponding image blocks in the target 
and reference images are compared based on the "proportion of occurrences" of the colors 
therein, thereby effecting a comparison of the histogram bins ("subdivisions") for corresponding 
blocks in each of the two images); 

processing the second image based on whether the first series of histogram subdivisions 
and the second series of histogram subdivisions have a similarity (column 6, lines 19-29: if the 
first and second series of histograms subdivisions are similar (as well as other corresponding 
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series pairs), then the reference image is identified as being similar to the target image, and is 
processed so that it is displayed in a list of similar images). 

Krishnamachari is silent to selecting the blocks ("subsets") of pixels ("image partitions") 
"based upon criteria related to a 3-D region of the object" and processing "the region" based on 
whether the pair of series of histogram subdivisions is similar. 

Moghaddam discloses a content-based image retrieval system, similar to that of 
Krishnamachari, in which database images that are substantially similar to a target image are 
retrieved. Moghaddam, like Krishnamachari, utilizes local color histograms in order to effect a 
comparison between a target and a reference image. 

Moghaddam also discloses the feature of allowing a user to specify regions of interest 
which serve as search criteria and lead to "a more powerful search engine." See § 1. As shown in 
figure 1, a user can select region(s) of interest corresponding to three-dimensional objects in a 
scene. Those regions of interest are then processed using local histogram measures, as shown in 
figure 2. The histograms of those regions are then matched to histograms of the reference images 
in order to find a sufficiently matching image. 

It would have been obvious to one of ordinary skill in the art at the time of the invention 
to modify Krishnamachari by Moghaddam to select the subsets of image partitions "based upon 
criteria related to a 3-D region of the object" and process "the region" based on whether the pair 
of series of histogram subdivisions is similar since Moghaddam discloses that selecting subsets 
of image partitions in a target image based on a region of a 3-D object according to a user's 
preference and then processing (e.g. retrieving and displaying) the region based on the similarity 
between histogram subdivisions of the target and reference images produces "a more powerful 
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search engine"; a user is allowed to specify arbitrary regions of objects to be retrieved rather than 
relying on a computer to specify the regions (see Abstract and § 1). 

Regarding claim 2, Krishnamachari discloses that the obtaining step comprises obtaining 
images of a scene as the object (figure 1 : "target image" and reference image(s) in the "image 
database" are images of scenes). 

Regarding claim 3, Krishnamachari discloses subdividing the first and second image into 
pixels (i.e. the images are digital images, which are divided into pixels). 

Regarding claim 5, Moghaddam discloses that only the pixels located in the region are 
selected (see figure 2(b)). 

Regarding claim 6, Krishnamachari discloses assigning an array value as the color value 
(column 4, lines 33-59: e.g. LUV array value). 

Regarding claim 7, Krishnamachari discloses assigning a 3-D array value (i.e. LUV is a 
3-D array). 

Regarding claim 8, Krishnamachari discloses that utilizing R, G, and B values is 
conventional (column 1, lines 35-45). 

Regarding claim 9, Krishnamachari discloses that the RGB values are typically 24 bits, 
so that the values of each color range from 0-255 (column 1, lines 35-45). 

Regarding claim 10, Krishnamachari discloses combining the R, G, and B values into a 
smaller number of agglomerate values (column 1, lines 55-64). 

Regarding claim 1 1, Krishnamachari discloses that combining portions of the first and 
second histogram subdivisions into a first and second series of histogram partitions is 
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conventional (column 1, lines 47-67: the 24-bit RGB values, representing millions of colors, are 
quantized into 64 colors so that the millions of histogram subdivisions for each image are 
combined to form 64 partitions -> in other words, portions of the first and second histogram 
subdivisions (i.e. different groups of the histogram bins) are combined into a series of histogram 
partitions (i.e. the groups each combine to from one of 64 partitions)). 

10. Claim 4 is rejected under 35 U.S.C. 103(a) as being unpatentable over Krishnamachari in 
view of Moghaddam, and further in view of U.S. Patent 4,985,856 by Kaufman et al. 
("Kaufman"). 

Regarding claim 4, Krishnamachari and Moghaddam are silent to selecting the subset 
based on criteria related to a voxel region of the object. 

Kaufman discloses a 3-D image storage and retrieval system. Rather than storing 
conventional 2-D image data, Kaufman's system is operative to store and retrieve 3-D image 
data composed of voxels. 

It would have been obvious to one of ordinary skill in the art at the time of the invention 
to modify Krishnamachari and Moghaddam by Kaufman to select a the subsets based on criteria 
related to a voxel region, rather than criteria related to a pixel region, since Kaufman discloses 
that it is desirable to represent images in 3-D with voxels and store and retrieve such images (see 
column 2, lines 56-66 and column 9, lines 45-52). 

1 1. Claim 15 is rejected under 35 U.S.C. 103(a) as being unpatentable over Krishnamachari 
in view of Moghaddam, and further in view of U.S. Patent 6,691,126 by Syeda-Mahmood. 
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Regarding claim 15, Krishnamachari and Moghaddam are silent to the object including a 
Lamertian surface. 

Syeda-Mahmood discloses an image processing system for locating objects in a video 
database. In particular, Sydea-Mahmood discloses methods by which Lambertian surfaces are 
modeled and retrieved in a video database (see column 6, lines 41-48). 

It would have been obvious to one of ordinary skill in the art at the time of the invention 
to modify Krishnamachari and Moghaddam by Syeda-Mahmood to choose the object to include 
a Lambertian surface, since Krishnamachari and Moghaddam disclose that any arbitrary object in 
an image may be obtained for retrieval purposes, and Syeda-Mahmood shows that it is 
conventional to retrieve objects that represent Lamertian surfaces. 

12. Claim 14 is rejected under 35 U.S.C. 103(a) as being unpatentable over Krishnamachari 
in view of Moghaddam, and further in view of "Color Indexing" by Swain et al. ("Swain"). 

Regarding claim 14, Krishnamachari is silent to comparing corresponding histogram 
subdivisions, as claimed. 

Swain discloses the conventional manner in which histograms are compared. At § 3.1.1, . 
Swain discloses that histograms are compared by comparing corresponding histogram 
subdivisions via a "min" function to see if each subdivision contains at least one pixel. 

It would have been obvious to one of ordinary skill in the art at the time of the invention 
to modify Krishnamachari and Moghaddam by Swain to compare histograms as claimed, since 
Swain shows that comparing corresponding histogram subdivisions to ascertain whether pixels 
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are present in each of the bins is a conventional technique for effecting a comparison of 
histograms. 

13. Claim 17 is rejected under 35 U.S.C. 103(a) as being unpatentable over Krishnamachari 
in view of Moghaddam, and further in view of U.S. Patent 6,71 1,288 by Kim et al. ("Kim"). 

Regarding claim 17, neither Krishnamachari nor Moghaddam discloses assigning a 
uniform region color to the entire region in the object. 

Kim discloses a method for designating a local representative color value for image 
regions in order to facilitate image retrieval. In particular, Kim discloses assigning uniform 
region colors to each of the image regions based on the local color histogram (see figure 2 A, 
block S52). Then, the local representative color of each region is used to select an optimal 
searching algorithm. 

It would have been obvious to one of ordinary skill in the art at the time of the invention 
to modify Krishnamachari and Moghaddam by Kim to assign the region a uniform color since 
Kim discloses that utilizing local representative colors (i.e. uniform region colors) allows an 
optimal search algorithm to be automatically ascertained (see column 3, lines 18-25). 

Allowable Subject Matter 

14. Claims 12, 13, 16, 18, and 20 are objected to as being dependent upon a rejected base 
claim, but would be allowable if rewritten in independent form including all of the limitations of 
the base claim and any intervening claims. 
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Regarding claim 12, Krishnamachari does not disclose or suggest that either of the first or 
second series of histogram partitions partially overlap. Rather, Krishnamachari suggests that the 
64 histogram partitions would necessarily be non-overlapping. See also the explanation for claim 
11. 

Regarding claims 16 and 20, Krishnamachari discloses the obtaining, subdividing, 
selecting, assigning, placing steps, as pertaining to a second reference image in the image 
database. Krishnamachari also discloses comparing the third series of histogram subdivisions of 
the second reference image to the first series of the target image. However, Krishnamachari does 
not disclose comparing the third series of histogram subdivision of the second reference image to 
the second series of histogram subdivisions of the first reference image. In Krishnamachari' s 
system, characteristics of the reference images are not compared to each other; they are each 
exclusively compared to the target image. 

Regarding claim 18, neither Krishnamachari nor Moghaddam disclose or suggest 
deciding whether to use the region based on the number of partitions, as claimed. 

Conclusion 

15. TfflS ACTION IS MADE FINAL. Applicant is reminded of the extension of time 
policy as set forth in 37 CFR 1 .136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within TWO 
MONTHS of the mailing date of this final action and the advisory action is not mailed until after 
the end of the THREE-MONTH shortened statutory period, then the shortened statutory period 



Application/Control Number: 09/987,9 1 8 



Page 13 



Art Unit: 2623 

will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 
CFR 1 . 1 36(a) will be calculated from the mailing date of the advisory action. In no event, 
however, will the statutory period for reply expire later than SIX MONTHS from the mailing 
date of this final action. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Colin M. LaRose whose telephone number is (703) 306-3489. If 
attempts to reach the examiner by telephone are unsuccessful, the examiner's supervisor, Amelia 
Au, can be reached on (703) 308-6604. The fax phone number for the organization where this 
application or proceeding is assigned is (703) 872-9306. 

Any inquiry of a general nature or relating to the status of this application or proceeding 
should be directed to the TC 2600 Customer Service Office whose telephone number is (703) 
306-0377. 



30 June 2005 
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